WIP: Port bash script naveen521kk to .travis.yml #1

stuaxo · 2020-08-19T13:48:26Z

Port WHL build script @naveen521kk to plain .travis.yml

Delete build script and move to .travis.yml

Minor changes:

Introduce "ARCH" variable for windows, x86 or x64
Rename PYVER to PYTHON_VERSION to be consistent with CAIRO_VERSION
Downloads are grouped with downloads from other platforms (in theory we could fail early here if they fail)
Files are downloaded to another folder (coverage was trying to check the python installation!)
Added the newer tests from newer pycairo

TODO
Having added the newer tests, test_typing fails.

naveen521kk · 2020-08-19T13:51:31Z

Question: Any necessity for it to be merged in master? Because I used the branch windows-wheels for the other PR. Also, I allowed maintainer to commit directly in that branch. SO you could directly commit if needed. Or if you want me to review the code make a PR to windows-wheels branch.
Cheers

stuaxo · 2020-08-19T14:03:36Z

Question: Any necessity for it to be merged in master? Because I used the branch windows-wheels for the other PR. Also, I allowed maintainer to commit directly in that branch. SO you could directly commit if needed. Or if you want me to review the code make a PR to windows-wheels branch.
Cheers

Nope, this was definitely my error ! In a bit of a rush now :)
It would be great if you can sanity check this... I think it works, but I haven't tested the whls for a couple of days (when I was using a modified version of your script).

stuaxo · 2020-08-19T14:05:20Z

I've got to be AFK for a while so can't wait to see tests pass, it's possible I've done something silly like bodged the merge with the latest version of your work.

naveen521kk · 2020-08-19T14:13:07Z

Currently I am busy I will look into this maybe by tomorrow.

naveen521kk

This was some quick questions I came up. Also, can we create a GitHub Action (As it is the only one I know has scheduled build) which checks for upstream for cairo version and other things? A python script would do. Just execute the py script and then create an issue if it not update. Any ideas about this?

.travis.yml

naveen521kk · 2020-08-19T15:13:06Z

.travis.yml

-      name: Build on Python 3.8.3 in Windows
-      env: PYVER="3.8.3" CAIRO_VERSION=1.17.2
+      name: Build on x64 Python 3.8.3 in Windows
+      env: ARCH=x64 CAIRO_VERSION=1.17.2 PYTHON_PACKAGE=python PYTHON_VERSION="3.8.3"


This can be updated to 3.8.5 as it is released now.

Suggested change

env: ARCH=x64 CAIRO_VERSION=1.17.2 PYTHON_PACKAGE=python PYTHON_VERSION="3.8.3"

env: ARCH=x64 CAIRO_VERSION=1.17.2 PYTHON_PACKAGE=python PYTHON_VERSION="3.8.5"

Good spot...

We can automate this, I spoke to dependencies.io who gave me an account for open source stuff + that can generate diffs and PRs for this sort of thing.

It's annoying nuget doesn't accept a query like 3.7.x and give the highest match, I wonder if chocolatey can do it?

You can use a query like 3.7 I think.

Also I just found that choco and nuget are one and the same thing in structure. Choco uses nugets structure.

Chocolatey has some extensions, I'm inclined to switch back to it, as it has some advantages.
Native support in Travis: one less thing to install, and Travis' cache support seems to include it.

You can force it to install for 32 or 64 bit as needed.

Semver: not supported now, but they have a ticket.
chocolatey/choco#1610

In the meantime I think we might be able to do semver support with a combination of powershell and the output of choco search or hit their API directly with cURL.

This would be good as we could specify the python version with 2 digit versions, not 3, so one less thing to support and patch.

We can pass - -x86 to install 32 bit versions, I haven't tried it yet.

That doesn't help much. Nuget python is build for CI ,but choco is using default py installer which may be troublesome. So better to use nuget for x86.

Fair enough, will stay on nuget, if there are questions around chocolatey.

Do you know what the issues with the default installer are?

I haven't used windows as my default OS for a while, well aware windows + python + C extensions can get pretty painful.

I kinda use windows as my default one. Maybe because I don't know C or something. I just know quite a few basic of python and got to know about this because of manim. Some video tool which use pycairo as it's base. I don't even have C compilers in my PC and that made me do this. But yeah Nuget x86 py build should be used or we should use something kinada docker or something but not required.

This can be easily fixed if we were to use GitHub Actions.

.travis.yml

naveen521kk · 2020-08-19T18:32:09Z

Also, can we create a GitHub Action (As it is the only one I know has scheduled build) which checks for upstream for cairo version and other things? A python script would do. Just execute the py script and then create an issue if it not update. Any ideas about this?

What do you think about this? @stuaxo

stuaxo · 2020-08-19T18:40:25Z

I can def do the github action to generate patch PRs, I have one I need to push upstream to cairo-windows that already does this, so fairly straightforward.

I'm a bit new to github actions so wasn't sure the best events for it to run on, so am thinking scheduled daily, and also available from a webhook (I imagine only @lazka will have the token for that).

As soon as someone merges the PR then I guess that will trigger a build and release (on github at least).

naveen521kk

Some suggetsions

.travis.yml

naveen521kk · 2020-08-20T15:40:08Z

About test failing I remember I read somewhere where lazka said about tee surface not implemented in windows or something like that. So its good to ping(@lazka) to help us on this or can we exempt these test on windows. @stuaxo

We can skip that test for windows see https://docs.pytest.org/en/latest/skipping.html#id1

Fix adding python to PATH Co-authored-by: Naveen M K <[email protected]>

@naveen521kk

Install wheel for all OSs, - via @naveen521kk Co-authored-by: Naveen M K <[email protected]>

stuaxo · 2020-08-20T21:02:56Z

Also, can we create a GitHub Action (As it is the only one I know has scheduled build) which checks for upstream for cairo version and other things? A python script would do. Just execute the py script and then create an issue if it not update. Any ideas about this?

What do you think about this? @stuaxo

Definitely doable, I think I wrote somewhere else we can do with dependencies.io and their deps tool.

I wonder which things we want to check on... if we can get rid of the need for an exact python build. then that leaves cairo-windows, at least until we make whls for other platforms.

naveen521kk · 2020-08-21T06:46:10Z

How about py 3.9 binary?

stuaxo · 2020-08-21T14:39:00Z

We're OK until October the 5th for 3.9
https://www.python.org/dev/peps/pep-0596/#schedule

naveen521kk · 2020-08-21T17:10:24Z

I will need to check and work something and will do in two days and later it can be merged.

naveen521kk · 2020-08-23T08:03:48Z

Now tests should be passing.

naveen521kk · 2020-08-23T08:21:23Z

@stuaxo We messed up something with x86 builds.

naveen521kk · 2020-08-23T09:10:52Z

@stuaxo If the tests pass this is ready to be merged.

stuaxo

I think we can change the test to check for minimum version to also check for Windows...

Do we know everything works OK with TeeSurface not enabled?

We might be able to just enable TeeSurface upstream, I don't have the dev environment setup to try this right now, but if it works we could always patch upstream and this will be fixed for good (it seems like msys has this enabled).

naveen521kk · 2020-08-23T18:03:23Z

it seems like msys has this enabled

I noticed about that. Maybe let's use that? Include the cairo.dll it generates to the .whl files. That would do. Also, I have tested this with the site which provided wheel that TeeSurface doesn't work on it also. Also, maybe it would be better to chat on gitter that these comments.

naveen521kk · 2020-08-23T18:09:18Z

Do we know everything works OK with TeeSurface not enabled?

I am such a noob here. I don't know C lol. So, I have no much experience in pycairo also.

naveen521kk · 2020-08-24T05:08:03Z

I got some weird ideas. @stuaxo

Why not move from Travis to GitHub Actions because Travis kinda sucks me ( I have more experience with Actions).
Not to depend on preshing repo instead use msys for our needs( There a gh-actions for that which makes it simple than Travis).

naveen521kk · 2020-08-24T18:21:43Z

Looks like I won't be able to spend much time on this in the upcoming week and I plan to take a long break. So I would be working on this PR next week.
Just tell me what to do it would be good. If I couldn't complete it then you(or anyone else) should be taking care of this 🙂. Also, I am adding you as collaborators to this fork. Please don't delete any release as some project depend on them.
Yes, I am running out of time and this would be the last week. Next, I won't be available.
CC
@stuaxo

stuaxo · 2020-08-24T19:45:28Z

I've got a similar time problem.. Will reply to this shortly though :)

stuaxo · 2020-08-24T20:21:30Z

I got some weird ideas. @stuaxo

* Why not move from Travis to GitHub Actions because Travis kinda sucks me ( I have more experience with Actions).

Yeah, definitely if @lazka is up for it.

* Not to depend on preshing repo instead use msys for our needs( There a gh-actions for that which makes it simple than Travis).

This makes sense, but I reckon we can probably be able to land this first, as it does seem to work.

We should verify there are no new issues using this when pygobject is already installed, then it should be good to go.

I'm also going to have less time, but should be able to help land the WHL support at least.

naveen521kk · 2020-08-26T18:34:18Z

Question: Does the whl available on https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycairo enables tee surface? We need to look into it.

naveen521kk · 2020-08-27T07:31:01Z

Does the whl available on https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycairo enables tee surface?

Yes it works there.

naveen521kk · 2020-08-27T08:24:02Z

@stuaxo If you could https://github.com/naveen521kk/pycairo/runs/1035437105?check_suite_focus=true#step:6:165 help me fix this I would be very happy and also it can be used by preshing/cairo-windows and use automatic releasing using Github actions. After that, maybe lets depend on then.

stuaxo · 2020-08-27T12:09:26Z

I'll have a look, I'm back @ work at the moment so won't be until later.

I've been looking at getting TeeSurface enabled on cairo-windows, which would mean we don't nee to change any tests
preshing/cairo-windows#7

@preshing has suggested a solution to get it working, but I won't have time until @ least later on,
If you have a VS2017 build environment it might be worth trying.

I started a PR there to generate make PRs when there are new releases of cairo, freetype etc update using dependencies.io
preshing/cairo-windows#10
I probably need to add some instructions on what setup is needed so it's easier for @preshing to land it.

stuaxo · 2020-08-27T12:11:26Z

Quick feedback: I can see it's building with VS2019, I don't really know the differences between 2017 + 2019, but @preshing told me that cairo-windows is currently built on 2017, not 19, so there may be some compatibility issue.

naveen521kk · 2020-08-30T04:30:38Z

Yes @stuaxo does https://github.com/naveen521kk/pycairo/runs/1046803166?check_suite_focus=true work? I am asking on tee surface did that work? If you any test environment download the wheels from artifact's and test it once locally. It that works we can tell @preshing to release it so. And yes tee surface problem will be over.
Edit: I have done something stupid for x86 builds. So please ignore that one instead use the one from the latest commit.

stuaxo · 2020-08-30T12:10:26Z

@naveen521kk I'm away from my computer, any idea why the x64 bit build fails?

All I tried was telling Travis to rebuild from my phone, think I need to verify from windows VM when in front of computer later.

naveen521kk

Here is what happens.

naveen521kk · 2020-08-30T12:27:48Z

.travis.yml

+  - if [ "$TRAVIS_OS_NAME" == "windows" ]; then $TMP/nuget install python -Version $PYTHON_VERSION -OutputDirectory $TMP/nuget-$ARCH; fi
+  - if [ "$TRAVIS_OS_NAME" == "windows" ]; then export INCLUDE="$TMP/cairo-windows-$CAIRO_VERSION/include"; fi
+  - if [ "$TRAVIS_OS_NAME" == "windows" ]; then export LIB=$TMP/cairo-windows-$CAIRO_VERSION/lib/$ARCH; fi
+  - if [ "$TRAVIS_OS_NAME" == "windows" ]; then export PATH=$TMP/nuget-$ARCH/python.$PYTHON_VERSION/tools/:$TMP/nuget-$ARCH/python.$PYTHON_VERSION/tools/Scripts:$PATH; fi


I found why it fails. Because the nuget package pythonx86 contains x86 build of python and it does pull x64 python and then later tries to build for x86 which makes is fail. :( Check my shell script.

Update cairo-windows version to "with-tee" tag.

Revert cairo-windows version, need to do this in front of computer with proper time :)

Broke version numbers as trying to type with 2 mins to spare :)

Fix cairo-windows URL

stuaxo · 2020-08-30T12:51:28Z

I need to stop trying to edit this at the same time as trying to get a 3 year old ready to get out of the house :) I'll come back to this when I can give it my full attention and stop adding duff commits :)

Don't worry, I will squash the commits before we push anything to the pycairo proper :)

stuaxo · 2020-08-30T20:36:53Z

I need to stop trying to edit this at the same time as trying to get a 3 year old ready to get out of the house :) I'll come back to this when I can give it my full attention and stop adding duff commits :)

Don't worry, I will squash the commits before we push anything to the pycairo proper :)

There's a pretty good chance it might be the when the work week starts (Tuesday it's a long weekend) when I can look at stuff properly, as it's almost all small person time, then just tiny bits of adult time during the weekend (and some small bits of coding time)!

naveen521kk · 2020-08-31T03:05:46Z

Maybe I will work if I get some time.

@stuaxo

Specified the variable `$PYTHON_PACKAGE` to install. cc. @stuaxo

naveen521kk · 2020-08-31T07:00:23Z

Any idea why https://travis-ci.com/github/naveen521kk/pycairo/jobs/379364608 fail? @stuaxo
Gonna rerun it.

stuaxo · 2020-08-31T10:58:21Z

Squashed and merged to pygobject/pycairo.

naveen521kk added 3 commits August 14, 2020 11:20

Try using nuget

6ddbd97

Make as single script

7366e95

Remove a line

8883cf1

stuaxo mentioned this pull request Aug 19, 2020

Build Windows wheels pygobject/pycairo#191

Closed

stuaxo changed the base branch from master to windows-wheels August 19, 2020 14:00

naveen521kk requested changes Aug 19, 2020

View reviewed changes

naveen521kk requested changes Aug 20, 2020

View reviewed changes

.travis.yml Show resolved Hide resolved

.travis.yml Outdated Show resolved Hide resolved

.travis.yml Outdated Show resolved Hide resolved

WIP: Port bash script naveen521kk to .travis.yml

895a850

stuaxo force-pushed the feature/port-to-tavis-yml branch from 75df715 to 895a850 Compare August 20, 2020 19:28

stuaxo and others added 2 commits August 20, 2020 21:52

Update .travis.yml

112a22e

Fix adding python to PATH Co-authored-by: Naveen M K <[email protected]>

Update .travis.yml

76a3b97

Install wheel for all OSs, - via @naveen521kk Co-authored-by: Naveen M K <[email protected]>

Merge branch 'windows-wheels' into feature/port-to-tavis-yml

a18151a

stuaxo force-pushed the feature/port-to-tavis-yml branch from 707b4bd to a18151a Compare August 20, 2020 21:42

Ignore windows from test

b4ca83f

stuaxo commented Aug 23, 2020

View reviewed changes

naveen521kk requested changes Aug 30, 2020

View reviewed changes

stuaxo added 4 commits August 30, 2020 13:33

Update .travis.yml

2bf0f5c

Update cairo-windows version to "with-tee" tag.

Update .travis.yml

0893722

Revert cairo-windows version, need to do this in front of computer with proper time :)

Update .travis.yml

7e36ce5

Broke version numbers as trying to type with 2 mins to spare :)

Update .travis.yml

f2b9591

Fix cairo-windows URL

naveen521kk added 2 commits August 31, 2020 12:20

Revert to original changes - Testing

447e8ae

Fix x86 build

d7727ca

Specified the variable `$PYTHON_PACKAGE` to install. cc. @stuaxo

stuaxo closed this Aug 31, 2020

	env: ARCH=x64 CAIRO_VERSION=1.17.2 PYTHON_PACKAGE=python PYTHON_VERSION="3.8.3"
	env: ARCH=x64 CAIRO_VERSION=1.17.2 PYTHON_PACKAGE=python PYTHON_VERSION="3.8.5"

WIP: Port bash script naveen521kk to .travis.yml #1

WIP: Port bash script naveen521kk to .travis.yml #1

Conversation

stuaxo commented Aug 19, 2020 • edited Loading

naveen521kk commented Aug 19, 2020

stuaxo commented Aug 19, 2020

stuaxo commented Aug 19, 2020

naveen521kk commented Aug 19, 2020

naveen521kk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stuaxo Aug 20, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

naveen521kk Aug 21, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

naveen521kk commented Aug 19, 2020 • edited Loading

stuaxo commented Aug 19, 2020

naveen521kk left a comment

Choose a reason for hiding this comment

naveen521kk commented Aug 20, 2020 • edited Loading

stuaxo commented Aug 20, 2020

naveen521kk commented Aug 21, 2020

stuaxo commented Aug 21, 2020

naveen521kk commented Aug 21, 2020

naveen521kk commented Aug 23, 2020

naveen521kk commented Aug 23, 2020

naveen521kk commented Aug 23, 2020

stuaxo left a comment

Choose a reason for hiding this comment

naveen521kk commented Aug 23, 2020 • edited Loading

naveen521kk commented Aug 23, 2020

naveen521kk commented Aug 24, 2020 • edited Loading

naveen521kk commented Aug 24, 2020

stuaxo commented Aug 24, 2020 • edited Loading

stuaxo commented Aug 24, 2020

naveen521kk commented Aug 26, 2020

naveen521kk commented Aug 27, 2020

naveen521kk commented Aug 27, 2020

stuaxo commented Aug 27, 2020

stuaxo commented Aug 27, 2020

naveen521kk commented Aug 30, 2020 • edited Loading

stuaxo commented Aug 30, 2020 • edited Loading

naveen521kk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stuaxo commented Aug 30, 2020

stuaxo commented Aug 30, 2020 • edited Loading

naveen521kk commented Aug 31, 2020

naveen521kk commented Aug 31, 2020 • edited Loading

stuaxo commented Aug 31, 2020

stuaxo commented Aug 19, 2020 •

edited

Loading

stuaxo Aug 20, 2020 •

edited

Loading

naveen521kk Aug 21, 2020 •

edited

Loading

naveen521kk commented Aug 19, 2020 •

edited

Loading

naveen521kk commented Aug 20, 2020 •

edited

Loading

naveen521kk commented Aug 23, 2020 •

edited

Loading

naveen521kk commented Aug 24, 2020 •

edited

Loading

stuaxo commented Aug 24, 2020 •

edited

Loading

naveen521kk commented Aug 30, 2020 •

edited

Loading

stuaxo commented Aug 30, 2020 •

edited

Loading

stuaxo commented Aug 30, 2020 •

edited

Loading

naveen521kk commented Aug 31, 2020 •

edited

Loading